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Abstract 

A  sensor  network  operates  on  an  infrastructure  of  sensing,  computation,  and  com¬ 
munication,  through  which  it  perceives  the  evolution  of  events  it  observes.  We 
propose  a  fusion-driven  distributed  dynamic  network  controller,  called  MDSTC, 
for  a  multi-modal  sensor  network  that  incorporates  distributed  computation  for  in- 
situ  assessment,  prognosis,  and  optimal  reorganization  of  constrained  resources 
to  achieve  high  quality  multi-modal  data  fusion.  For  arbitrarily  deployed  sensors, 
certain  level  of  data  quality  cannot  be  guaranteed  in  sparse  regions.  MDSTC  reallo¬ 
cates  resources  to  sparse  regions;  reallocation  of  network  resources  in  this  manner 
is  motivated  by  the  fact  that  an  increased  density  of  sensor  nodes  in  a  region  of 
interest  leads  to  better  quality  data  and  enriches  the  network  resilience.  Simulation 
results  in  NS-2  show  the  effectiveness  of  the  proposed  MDSTC.  1 
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1.  Introduction 


A  distributed  sensor  network  consists  of  many  low-cost  mobile  or  fixed  sensor 
nodes  that  are  networked  to  observe  spatial-temporal  distributed  events.  Obser¬ 
vations,  decisions  or  estimates  made  by  individual  sensors  are  fused  together  to 
create  a  composite  view  of  the  situation.  The  sensor  nodes  must  dynamically  col¬ 
laborate  to  fuse  information  in  the  vicinity  of  an  arbitrary  critical  event  to  optimize 
its  identification,  classification,  and  tracking  in  operational  environments.  The  sen¬ 
sor  nodes  and  the  network  they  form  are  resource-constrained  in  terms  of  network 
bandwidth,  the  lifetime  of  the  node’s  battery,  sensing  range  and  communication  ra¬ 
dius.  Consequently,  it  is  important  to  dynamically  control  the  available  resources 
to  deliver  acceptable  sensor  fusion  performance  and  possibly  to  maximize  this  per¬ 
formance,  measured  in  terms  of  data  accuracy  for  fusion.  The  techniques  described 
combine  distributed  control  and  self-adaptation  of  the  network  topology  to  the  re¬ 
quirements  of  the  application  running  at  the  highest  level  at  each  sensor  node,  and 
can  be  applied  to  solve  many  problems  in  military,  homeland  security,  medicine, 
civil  engineering,  bathymetry,  terrain  mapping,  etc..  Previous  research  efforts  [1] 
focused  on  human-controlled  or  human-guided  sensor  networks.  In  this  paper  we 
consider  networks  that  are  autonomous;  there  is  no  human  intervention  involved  in 
designing  or  organizing  the  network. 

Heterogeneous  sensor  networks  have  been  extensively  studied  for  the  purpose 
of  classifying  and  tracking  mobile  targets,  by  obtaining  an  accurate  estimation  of 
the  sensed  event(s)  locations  based  on  the  sensors’  locations  [2],  measured  signal 
strength,  and  the  quality  of  the  raw  data  [3,  4,  5,  6],  When  a  multi-modal  sensor 
network  is  used  to  track  moving  targets,  the  critical  resource  optimization  param¬ 
eters  are  power  consumption,  coverage,  communication,  and  data  fusion.  Major 
sources  of  wasted  energy  are  communication  collisions,  overhearing,  idle  listen- 
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ing,  and  control  overhead  [7].  Idle  listening  can  be  minimized  by  an  appropriate 
sleep-and-awake  schedule  for  nodes  [8,  9,  10,  11].  Collisions  can  be  minimized 
by  scheduling  the  local  transmission  [12]  and  overhearing  can  be  minimized  by  re¬ 
ducing  the  transmission  power  [13].  To  optimize  the  region  coverage,  various  fault 
tolerant  schemes  for  the  network  topology  are  researched  in  order  to  guarantee  that 
a  certain  area  is  covered  [14,  15,  16,  17].  To  optimize  communication,  the  network 
capacity  is  to  be  maximized  and  the  network  delay  is  to  be  minimized  [18].  To 
optimize  data  fusion,  interference  is  to  be  reduced  by  scheduling  nodes’  transmis¬ 
sion  [19],  higher  fidelity  of  raw  data  is  to  be  obtained  by  moving  sensors  closer 
to  observed  event(s)  [16],  high  fidelity  compressed  data  is  to  be  obtained  from  the 
raw  data  [20],  and  the  delivery  rate  of  fused  data  is  to  be  increased  [21]. 

When  a  mobile  target  approaches  a  sensor  node  and  the  stimulus  at  the  node 
is  above  a  given  threshold,  an  event  is  said  to  have  been  detected.  However,  if 
the  threshold  is  not  reached  then  the  event  is  labeled  as  undetected  and  ignored. 
When  an  event  is  detected  by  a  sensor  node,  the  node  broadcasts  a  message  con¬ 
taining  the  event  description  and  the  event  data.  Since  the  events  are  assumed  to 
be  localized  in  time  and  space,  a  burst  of  communication  will  occur  in  a  small  geo¬ 
graphic  region  surrounding  the  target.  So,  instead  of  analyzing  the  topology  of  the 
entire  network,  we  concentrate  on  the  topology  of  a  relatively  small  geographical 
region  around  the  mobile  target  where  nodes  that  are  geographically  near  observe 
an  event  of  interest  and  form  multi-modal  (heterogeneous)  clusters  that  together 
support  a  specified  level  of  multi-modal  data  fusion.  We  present  MDSTC,  a  dis¬ 
tributed  network  controller  for  a  multi-modal  sensor  network,  responsible  for  clus¬ 
tering  nodes  in  the  space-time  vicinity  of  a  moving  object,  improving  data  fusion 
by  adapting  the  network  topology  (i.e.,  changing  the  location  of  selected  mobile 
nodes  and  the  transmission  range  of  selected  nodes),  localizing  and  tracking  the 
object.  MDSTC  is  an  adaptation  of  the  dynamic  space-time  clustering  (DSTC) 
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protocol  [22,  23,  24,  25]  to  the  sensors’  heterogeneity  and  mobility;  it  has  the  en¬ 
hanced  capability  to  coordinate  mobile  nodes  within  a  cluster  to  move  to  nearby 
locations  in  order  to  lower  interference,  avoid  field  obstructions,  and  cover  sparse 
regions.  Thus  the  topology  of  the  cluster  self-adapts  to  achieve  better  data  fusion. 

During  the  data  fusion  (necessary  for  tracking  moving  events),  two  modes  of 
intra-cluster  communication  are  supported:  between  the  same  modality  sensors 
and  between  sensors  of  complementary  modalities.  In  order  to  minimize  the  local 
network  communication  that  leads  to  reduced  power  consumption  and  network 
bandwidth,  we  also  describe  a  novel  multicast  group  management  protocol  that 
handles  multicast  groups  between  same-modality  and  complementary  sensors. 

The  paper  is  organized  as  follows.  In  Section  2  we  present  the  network  archi¬ 
tecture,  the  system  model,  and  the  communication  optimization  through  multicast. 
The  multi-modal  data  fusion  at  the  nodes  observing  an  event  of  interest  is  pre¬ 
sented  in  Section  3.  In  Section  4  we  present  the  multi-modal  cluster  formation 
and  the  dynamic  adaptation  of  the  multi-modal  cluster  to  improve  data  fusion.  In 
Section  5  we  show  how  the  proposed  multi-modal  network  controller  improves  the 
quality  of  the  multi-modal  data  fusion  and  we  present  the  results  of  the  experiments 
conducted  in  our  laboratory.  Section  6  presents  conclusions  and  future  work. 

2.  Network  Architecture  and  System  Model 

An  event  is  a  dynamical  process  that  is  observed  by  sensors  of  various  modal¬ 
ities.  The  time-series  data  observed  by  a  sensor  of  a  certain  modality  can  be  com¬ 
pressed  to  a  probabilistic  finite  state  automaton  (PFSA)  with  an  a  priori  fixed  al¬ 
phabet  [26],  which  depends  strictly  on  the  modality  of  the  sensor.  The  PFSA  is 
designed  to  extract  the  maximum  semantic  information  from  the  raw  sensor  data, 
while  at  the  same  time  it  compresses  the  sensory  data  to  minimize  the  network 
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bandwidth  requirements.  At  each  node  the  data  is  handled  in  a  module  called  In¬ 
formation  Space  and  the  network  communication  issues  and  adaptation  decisions 
are  addressed  by  a  separate  module  called  Network  Design  Space  [27,  28];  the 
system  model  at  a  node  is  presented  in  Figure  1 . 


Network  Design  Space  (NDS)  Information  Space  (IS) 


Traffic  Analyzer 

K 

Application-dependent  Computations 
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Figure  1 :  System  model  for  a  sensor  node  (independent  of  the  sensor  modality) 

DSTC  [22]  protocol  works  for  single-modality  wireless  networks  and  it  has 
five  basic  operations:  creating  a  cluster,  disbanding  a  cluster,  adding  a  node  to 
a  cluster,  removing  a  node  from  a  cluster,  and  moving  the  data  from  the  cluster 
head  to  another  cluster  member.  We  adapted  DSTC  to  multi-modal  mobile  sensors 
and  we  call  the  new  algorithm  as  the  multi-modal  dynamic  space-time  clustering 
(MDSTC).  MDSTC  is  part  of  NDS  and  is  responsible  for  clustering  nodes  in  the 
space-time  vicinity  of  a  moving  object,  improving  data  fusion,  adapting  the  net¬ 
work  topology  by  moving  mobile  sensors  to  dynamically  computed  locations  in 
order  to  cover  sparse  regions  and  lower  communication  interference,  localizing 
and  tracking  the  object  that  has  triggered  the  composite  pattern. 

As  DSTC,  MDSTC  allocates  the  sensor  network  resources  only  in  the  space- 
time  vicinity  of  an  event  or  object,  resources  that  belong  to  multi-modal,  resource- 
constrained,  mobile  sensor  nodes.  It  also  dynamically  alters  the  topology  of  each 
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cluster  formed  around  an  event  by  allocating  and  de-allocating  resources  (i.e.,  mov¬ 
ing  sensors  in  or  out  of  a  cluster)  in  order  to  maintain  a  specified  level  of  fusion 
performance.  The  space-time  vicinity  of  an  event  [29,  22,  24]  is  defined  as  the  set 
of  all  space-time  points  that  are  close  to  the  space-time  point  (xo,  to)  within  a  time 
and  space  boundary: 

r)(x0,t0,Ax,At)  =  {(x,i)|||x  -  x0||  <  Ax  A  ||t-t0||  <  At}.  (1) 
The  quantities  Ax  and  At  define  the  size  of  the  neighborhood.  Multi-modal  data 
that  was  measured  within  a  distance  Ax  around  xo  and  within  the  time  interval 
to  ±  At  will  be  used  for  data  fusion  and  the  sensors  within  this  neighborhood  will 
be  considered  for  a  multi-modal  space-time  cluster. 

A  distributed  network  of  sensors  achieves  high  performance  by  exploiting  sen¬ 
sor  data  fusion.  Sensors  of  various  modalities  group  together  to  form  dynamic 
multi-modal  clusters;  their  sensed  data  (of  various  modalities)  is  fused  locally  at 
each  node.  The  clustering  of  multi-modal  sensors  is  done  in  two  stages. 

Stages  1  and  2  of  data  fusion  entail  the  following  type  of  communication.  In 
Stage  1 ,  same-modality  sensor  nodes  located  within  the  sensing  radius  of  the  sensed 
event  need  to  inform  one  another  about  the  minimum  semantic  distance  between 
the  PFSA  observed  by  a  sensor  node  and  some  reference  PFSA  from  the  library. 
The  minimum  semantic  distance  argument  translates  into  the  best  match  argument 
between  an  observed  PFSA  pattern  and  the  set  of  reference  patterns.  Thus,  during 
Stage  1,  the  sensor  nodes  need  only  to  communicate  with  like-modality  sensors. 

Stage  2  focuses  on  constructing  the  composite  PFSA  that  has  the  best  possible 
composite  metric.  The  composite  PFSA  is  made  up  of  PFSA  from  all  sensors  of 
complementary  sensing  modalities.  Consequently,  during  Stage  2  only  communi¬ 
cation  between  sensors  of  different  sensing  modalities  is  needed.  In  the  sequel, 
we  describe  a  novel  multicast  grouping  algorithm  that  can  substantially  reduce  the 
communication  among  sensors,  as  well  as  computation  at  nodes. 
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Let  S  be  the  set  of  sensors  in  a  given  region  and  let  =  |«S|.  For  any  given 
sensing  modality  s,  the  set  S  can  be  partitioned  into  Ss,  the  subset  of  sensors  that 
have  the  same  sensing  modality  s,  and  S-,s  =  S  \  Ss,  the  subset  of  nodes  with  the 
sensing  modality  complementary  to  s ;  let  ks  =  Ss  \ . 

During  Stage  1 ,  all  nodes  in  S  need  to  broadcast  their  semantic  distance  among 
themselves;  the  message  will  be  processed  by  an  application  entity  present  in  the 
application  layer  of  the  protocol  stack  of  each  node.  Flowever,  this  broadcast  mes¬ 
sage  will  have  relevance  only  for  the  |fcs|  (<<  \k\)  sensor  nodes.  The  application 
entities  of  the  remaining  \k  —  ks\  sensors  of  complementary  sensing  modality  will 
however,  receive,  process  and  eventually  discard  such  messages,  resulting  in  un¬ 
necessary  processing  cycles  and  consumption  of  limited  battery  power. 

Stage  2  deals  with  the  formation  of  composite  PFSA  at  each  node  of  some 
modality  s,  which  are  made  up  of  individual  PFSA  provided  by  sensors  of 
sensing  modalities  in  addition  to  the  PFSA  constructed  by  the  node.  Therefore 
in  Stage  2,  a  node  from  <S-5  of  sensing  modality  s'  /  s  needs  to  send  its  PFSA 
only  to  a  node  of  sensing  modality  s;  this  is  the  converse  of  Stage  1.  After  a  node 
i  6  S^s  broadcasts  its  PFSA  to  all  the  nodes  in  5,  such  a  message  will  be  relevant 
to  only  the  nodes  in  <S.s;  the  remaining  k  —  ks  —  1  nodes  will  waste  their  processing 
cycles  in  receiving  and  then  discarding  such  a  message.  The  number  of  wasted 
processing  cycles  can  be  quite  substantial,  since  a  packet  meant  for  an  applica¬ 
tion  has  to  traverse  all  the  protocol  layers,  before  it  is  accepted  or  thrown  away 
by  the  application  layer.  Without  multicasting,  the  amount  of  CPU  cycles  used  by 
a  single  sensor  node  for  receiving  and  subsequently  rejecting  a  packet  is  equal  to 
C 'physical  layer  T  ^ MAC  layer  T  Cnnk  layer  T  ^network  layer •  ^Aith  multicasting,  in 
comparison  the  number  of  CPU  cycles  wasted  by  a  single  sensor  node  for  receiv¬ 
ing  and  subsequently  rejecting  a  packet  is  equal  to  CphySicai  layer  +  CMac  layer, 
since  an  IP  multicast  address  gets  embedded  within  the  MAC  address  assigned  to 
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the  network  interface.  Consequently,  packets  whose  destination  address  does  not 
correspond  either  to  the  unicast  address  or  to  one  of  the  multicast  addresses  (joined 
by  this  node)  are  filtered  out  by  the  MAC  layer  itself,  thus  saving  CPU  cycles  and 
energy  for  not  having  to  process  the  packet  in  the  link  and  network  layers. 

Let  us  consider  a  fully  connected  sensor  network  with  Acoustic  (A),  Visual 
(V),  and  Chemical  (C)  sensors,  n  of  each  modality.  During  Stage  1,  an  A-modality 
sensor  needs  to  communicate  with  the  rest  of  the  A-modality  sensors  only  to  ex¬ 
change  the  semantic  distance  metric  information.  This  implies  that  the  remaining 
2 n  sensors  waste  their  processing  cycles  when  A-modality  broadcast  packets  tra¬ 
verses  their  protocol  stacks.  During  Stage  2,  the  selected  cluster  head  (let  us  say 
an  A-modality  sensor)  needs  to  communicate  only  with  the  non-A  modality  sen¬ 
sors,  since  the  purpose  of  this  communication  is  to  create  a  composite  PFSA.  This 
implies  that,  when  V-  and  C-modality  sensors  broadcast  their  PFSA,  only  the  A- 
modality  cluster  head  has  to  take  cognizance  of  these  broadcast  packets.  A  naive 
processing  of  the  broadcast  packets  would  require  remaining  the  3n  —  2  nodes  to 
process  and  then  discard  such  packets.  To  address  this  waste,  we  describe  a  novel 
multicast  group  formation  technique  that  replaces  the  broadcast  with  the  multicast, 
thus  eliminating  the  waste  described  earlier.  It  turns  out  that  the  minimum  number 
of  multicast  groups  required  to  support  Stages  1  and  2  is  independent  of  the  number 
of  sensor  nodes,  and  it  depends  only  on  the  number  of  modalities  (Lemma  1). 

Lemma  1.  Given  a  sensor  network  with  n  nodes  and  rri  sensing  modalities,  the 
minimum  number  of  distinct  multicast  groups  required  to  support  Stages  1  and  2  is 
2m.  Out  of  these  2m  groups,  each  sensor  node  needs  to  join  only  m+  1  groups. 

Proof.  During  Stage  1,  at  least  one  node  of  some  modality  X  needs  to  send 
its  PFSA  to  sensors  of  the  same  modality  and  it  does  so  by  joining  a  group  Gx- 
Thus  the  minimum  number  of  multicast  groups  to  be  created  during  Stage  I  is  rn 


since  each  modality  is  represented  by  at  least  one  group. 

During  Stage  2,  a  node  of  some  modality  X  needs  to  send  its  PFSA  to  the 
sensor  nodes  of  non- A  modalities  and  it  does  so  by  joining  another  group  C-,x  that 
contains  sensors  of  modalities  other  than  X .  Also,  all  nodes  of  non- A'  modalities 
will  also  need  to  join  the  G^x  group  to  be  able  to  receive  a  PFSA  from  a  X- 
modality  sensor.  Thus  we  need  to  create  m  additional  groups  in  order  to  support 
Stage  2.  In  all,  we  require  a  minimum  of  2m  distinct  multicast  groups.  □ 

As  an  example,  for  the  3-modality  network  described  earlier  six  multicast 
groups,  Ga,  Gc,  Gy,  G^a,  G^c,  and  G^y,  need  to  be  formed.  During  Stage  1,  a 
sensor  node  of  modality  i  needs  to  communicate  with  sensors  of  the  same  modality. 
This  communication  takes  place  on  a  multicast  group  Gi  (see  Table  1(a)).  During 
Stage  2,  a  sensor  node  of  modality  i  needs  to  communicate  with  sensor  nodes  of 
modality  j  (j  /  i).  This  communication  occurs  on  the  multicast  group  given  by 
the  entry  (j,  i )  (of  row  j  and  column  i)  of  the  Table  1(6). 


Modality 

A 

C 

V 

A 

G^c 

G-,v 

C 

G-a 

G~,v 

V 

G^a 

G^c 

Modality 

A 

C 

V 

A 

Ga 

C 

Gc 

V 

Gy 

(a)  Groups  for  Stage  1  (6)  Groups  for  Stage  2 

Table  1:  Multicast  groups  for  multi-modal  data  fusion 

Each  sensor  of  modality  A  will  have  to  communicate  using  four  multicast 
groups,  Ga,  G~,a,  G~,c,  and  G~,y,  in  order  to  form  a  composite  pattern  of  in¬ 
terest.  In  general,  when  there  are  m  modalities,  each  sensor  needs  to  use  m  +  1 
multicast  groups  out  of  a  total  of  2m  multicast  groups  that  need  to  be  created, 
which  is  independent  of  the  number  of  the  sensors. 
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3.  Information  Space  -  Multi-modal  Data  Fusion 


When  a  mobile  target  approaches  a  sensor  node  and  the  stimulus  at  the  node 
is  above  a  given  threshold,  an  event  is  said  to  have  been  detected.  However,  if  the 
threshold  is  not  reached  then  the  event  is  labeled  as  undetected  and  ignored.  When 
an  event  is  detected  by  a  sensor  node,  the  node  broadcasts  a  message  containing  the 
event  data.  The  time-series  data  observed  by  a  sensor  of  a  certain  modality  can  be 
compressed  to  a  probabilistic  finite  state  automaton  (PFSA)  over  an  a  priori  fixed 
alphabet  [26].  The  PFSA  is  designed  to  extract  the  maximum  semantic  information 
from  the  raw  sensor  data,  while  at  the  same  time  it  compresses  the  sensory  data  to 
minimize  the  network  bandwidth  requirements. 

Multi-modal  data  fusion  is  done  at  individual  nodes  in  the  Information  Space 
(IS)  module.  During  the  training  phase  we  identify  and  store  a  number  of  PFSA 
of  interest,  which  we  call  subpatterns,  in  the  IS  of  a  sensor  node.  An  example  of 
such  a  subpattern  is  the  pressure  signature  of  a  car  passing  by  a  pressure  sensor 
(Section  3.1).  In  the  operational  phase,  when  an  event  is  observed  by  the  sensor, 
the  IS  at  the  sensor  node  will  construct  a  PFSA  based  on  the  time-series  data  from 
the  sensing  unit.  The  constructed  PFSA  will  be  compared  against  the  library  of 
subpatterns  and  a  semantic  distance  [20]  will  be  derived  from  each  of  the  existent 
subpatterns.  If  the  constructed  PFSA  has  a  semantic  distance  less  than  a  given  e 
with  regards  to  a  particular  subpattern,  we  conclude  that  the  sensor  has  detected 
that  subpattern,  and  we  have  a  subpattern  match. 

After  the  IS  matches  a  computed  PFSA  to  a  specific  subpattern,  it  sends  to  the 
network  controller  the  message  subpatternObserved  that  contains  the  subpattern 
ID  and  the  metric  which  is  the  semantic  distance  between  the  observed  subpattern 
and  the  subpattern  stored  in  the  databank.  The  exact  time  when  the  message  is  sent 
by  the  IS  depends  on  the  sensor  modality,  since  the  IS  has  to  collect  enough  data  in 
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order  to  construct  a  PFSA.  On  receiving  the  message  subpatternObserved  from  the 
IS,  MDSTC  appends  to  it  some  control  information  such  as  the  node  position,  node 
ID,  the  time,  and  broadcasts  it  as  the  message  Subpattem.  In  this  way  the  neigh¬ 
boring  nodes  are  notified  about  the  observed  subpattern.  The  messages  Subpattem 
can  be  used  for  target  localization  (see  [30]). 

Based  on  the  observed  subpattern  and  received  subpatterns  from  other  nodes, 
the  IS  will  try  to  decide  on  a  composite  pattern.  Namely,  at  each  node,  the  data 
received  from  sensors  of  different  modalities  is  fused  locally  in  order  to  generate 
patterns  of  interest  which  we  call  composite  patterns.  If  successful,  the  IS  notifies 
MDSTC  about  the  decided  composite  pattern. 

A  composite  pattern  is  the  Cartesian  product  of  subpatterns  in  each  modality.  If 
we  have  M  >  1  different  modalities,  a  composite  pattern  P  =  { P 1 .  P2, . . . ,  PM} 
is  a  M- tuple  of  subpatterns,  one  subpattern  for  each  modality.  At  each  node,  the 
Information  Space  stores  a  set  of  composite  patterns  of  interest,  P,  in  a  so-called 
databank.  The  databank  contains,  for  each  stored  pattern  of  interest  and  for  each 
individual  subpattern,  the  minimum  number  of  sensors  of  that  modality  that  are 
required  to  observe  the  subpattern  NPj  =  (j)j,i,Pj,2,  ■  ■  ■  ,Pj,M)>  Pj,i  P  1,  V  1  < 
l  <  M.  A  pattern  of  interest  has  an  associated  maximum  lifetime  that  represents 
the  period  during  which  the  pattern  is  observable. 

3.1.  Constructing  a  Subpattem 

To  construct  subpatterns  in  modality  m,  a  sensing  model  for  modality  m  is 
assumed.  We  show  next  how  subpatterns  are  constructed  for  pressure  sensors;  the 
pressure  sensor  model  in  this  case  is  omni-directional,  but  this  is  not  a  requirement 
for  other  modalities.  The  data  reading  of  a  pressure  sensor  changes  as  a  moving 
object  passes  by  (Figure  2(a));  when  a  mobile  object  approaches  the  pressure  sen¬ 
sor,  different  paths  of  the  target  determine  different  time  series  data  for  the  sensor. 
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Let  d  be  the  minimum  distance  between  the  object  and  the  sensor  at  any  moment  of 
time  (Figure  2(6));  the  solid  line  corresponds  to  the  subpattern  (trained  PFSA  when 
d  =  0),  which  corresponds  to  the  time  series  data  (solid  curve)  in  Figure  2(c).  The 
PFSA  constructed  from  the  time-series  data  starting  when  the  target  was  at  distance 
d  =  1  meter  to  the  sensor  is  shown  as  a  dotted  curve  in  Figure  2(6). 


(a) 


(b) 


(c) 


Figure  2:  Semantic  distance  for  pressure  sensors 


When  the  object  is  at  the  same  location  as  the  sensor  we  are  guaranteed  that  we 
have  a  perfect  subpattern  match.  This  coincides  with  the  concept  of  closest  point  of 
approach  ( CPA)  events  [31,  32,  33].  In  other  terms,  when  the  object  passes  closest 
to  the  sensor  node,  the  sensor’s  signal  reaches  a  peak  and  the  event  is  signaled.  The 
captured  event  is  called  a  CPA  event.  The  signal  peak  of  the  CPA  event  is  directly 
related  to  the  distance  between  the  object  and  the  sensor  because  the  signal-to- 
noise  ratio  of  sensing  decreases  with  the  distance.  The  closer  the  target  is  to  the 
sensor,  the  higher  the  signal  peak  of  the  CPA  event  is.  The  difference  between  two 
time  series  data  is  reflected  in  the  so-called  semantic  distance  e. 

For  magnetic  sensors,  the  subpatterns  are  constructed  exactly  the  same  way  as 
for  the  pressure  sensors.  Flowever,  the  subpatterns  of  video  sensors  are  constructed 
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differently  and  they  do  not  incur  CPA  events.  Therefore  the  PFSA  constructed  by 
the  video  sensors  will  only  be  used  for  pattern  classification  but  not  for  estimating 
the  event  location. 

4.  Network  Design  Space 

MDSTC  is  part  of  the  Network  Design  Space  (NDS)  module;  it  has  has  several 
responsibilities:  adjust  the  transmission  power  based  on  the  battery  power,  com¬ 
munication  with  other  nodes,  maintain  and  provide  statistics  of  this  communication 
to  the  IS  module,  cluster  multi-modal  sensor  nodes  in  the  space-time  vicinity  of  an 
event,  identify  sensor  nodes  to  be  included  or  excluded  from  the  clusters,  request 
sensor  nodes  to  move  in  order  to  improve  the  quality  of  the  data  fusion  or  to  cover 
sparse  regions  in  which  events  are  expected,  coordinate  the  detection  of  events  and 
correlate  events  based  on  spatial-temporal  occurrence.  The  precluster  and  cluster 
formation  of  DSTC  are  adapted  in  MDSTC  to  the  heterogeneity  of  the  sensors; 
this  adaptation  is  straightforward.  To  improve  the  quality  of  data  needed  for  de¬ 
ciding  on  a  composite  pattern  and  application-dependent  computations  (such  as 
estimating  the  target  location,  velocity,  and  predicting  its  trajectory),  the  cluster 
also  adapts  its  topology  to  the  requirements  of  the  IS  (see  Section  4.1)  by  adding 
selected  sensors  (Sections  4.2  and  4.4)  in  specific  locations  (Section  4.3). 

Each  multi-modal  cluster  has  an  associated  maximum  lifetime.  When  the  max¬ 
imum  lifetime  expires,  the  cluster  is  disbanded  by  the  cluster  head.  Thus  all  cluster 
members,  including  the  cluster  head,  become  free.  The  cluster  head  can  also  dis¬ 
band  the  cluster  when  its  battery  power  falls  below  a  certain  threshold. 

When  the  mobile  target  moves  through  the  sensor  network,  a  node  other  than 
a  cluster  head  may  observe  the  pattern  better.  Since  the  cluster  head  is  selected 
as  the  node  that  currently  observes  the  event  the  best,  we  may  need  make  that 
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node  as  the  cluster  head.  In  this  case,  we  may  need  to  disband  the  current  cluster 
and  form  another  cluster.  The  local  network  stability  may  suffer  if  we  create  and 
disband  clusters  as  soon  as  nodes  observe  better  than  the  current  cluster  heads.  We 
then  take  a  lazy  approach,  to  maintain  stability:  we  allow  the  cluster  to  live  for 
at  least  some  period  of  time,  that  we  call  minJifetime.  During  the  minJifetime, 
the  cluster  head  does  not  change  due  to  a  better  composite  metric  for  the  same 
composite  pattern.  Once  the  minJifetime  expires  and  before  maxJifetime  expires, 
it  is  possible  for  a  cluster  member  or  a  non-cluster  member  to  become  a  cluster 
head  and  force  the  current  cluster  head  to  disband  the  cluster. 

4. 1 .  Dynamically  Changing  the  Local  Topology 

We  recall  that  the  IS  of  the  node  knows  a  priori  the  number  of  nodes  of  each 
modality  needed  to  distinguish  between  subpatterns  or  composite  patterns.  Sensors 
need  to  be  moved  in  two  distinct  cases.  One  case  is  when  the  IS  of  the  node 
determines  that  it  has  too  little  information  to  properly  identify  either  a  uni-modal 
or  a  composite  pattern  of  interest,  or  it  needs  better  quality  data  for  composite  data 
fusion.  Another  case  is  when  the  IS  of  a  node  is  notified  that  the  trajectory  of  a 
mobile  target  will  intersect  its  sensing  coverage  region  and  it  has  determined  that 
there  are  not  enough  sensors  of  certain  modality  within  the  node’s  communication 
region  to  be  able  to  do  data  fusion. 

In  either  case,  the  IS  sends  the  message  moreData  to  the  MDSTC  with  the 
sensor  modality  needed,  the  number  of  extra  sensors  of  that  modality  needed,  and 
the  desired  region  where  the  sensors  are  needed  (the  center  and  the  radius).  The 
MDSTC  broadcasts  the  message  addNode  appended  with  the  node  ID,  the  node’s 
physical  location  or  coordinates,  the  sensor  modality  needed,  and  the  desired  re¬ 
gion.  The  number  of  sensors  needed  of  that  modality  is  stored  by  the  MDSTC. 

On  receiving  the  message  addNode'. 
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-  If  the  sensor  is  of  the  modality  requested  by  the  requester  node  and  the  requester 
is  further  away  from  the  center  of  the  desired  region,  then  drop  the  message.  -  If 
the  node’s  modality  is  not  the  same  as  the  requested  one  and  the  node’s  location  is 
not  far  from  the  desired  location,  then  broadcast  the  message  further. 

-  If  the  node’s  modality  is  the  same  as  the  requested  one  then  the  message  is  sent 
to  the  IS  for  further  processing  (see  Section  4.2)  and  may  agree  to  participate.  We 
note  that  the  nodes  that  have  agreed  to  the  request  have  to  move  within  to  the 
requester’s  transmission  range. 

Among  the  sensors  that  have  responded  positively  to  the  request,  the  MDSTC 
of  the  requester  will  decide  where  they  will  be  placed  (see  Section  4.3)  and  which 
of  them  will  be  asked  to  move  (see  Section  4.4). 

4.2.  When  a  Node  Decides  to  Move 

The  decision  of  moving  to  a  specific  region  is  done  in  the  IS.  We  note  that,  if 
the  node  is  not  within  the  communication  range  of  the  requester  node,  the  node 
has  to  move  into  the  requested  region  in  order  to  participate  in  data  fusion  and 
receive  subsequent  messages.  If  the  node  is  capable  of  moving  and  is  not  part  of 
any  cluster,  then  the  IS  estimates  the  cost  of  moving  per  unit  distance 

cpud  =  1  /{P  ■  p) 

based  on  the  node’s  power  P  (the  more  power,  the  lower  the  cost)  and  the  density 
p,  namely  how  many  sensors  of  the  same  modality  exist  within  the  node’s  commu¬ 
nication  range  (the  lower  the  density,  the  higher  the  cost).  This  value  together  with 
the  speed  speed  which  is  the  average  speed  the  sensor  can  move  and  the  requester 
ID  is  sent  as  the  message  Available  to  the  MDSTC.  When  the  MDSTC  receives 
the  message  Available  from  the  IS,  it  appends  the  node  ID  and  the  node  position 
and  the  message  is  then  sent  to  the  neighbor  from  which  the  message  addNode  has 
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been  received.  On  receiving  the  message  Available  not  addressed  to  itself,  a  node 
forwards  it  to  the  node  from  which  the  message  addNode  has  been  received.  On 
receiving  the  message  Available  addressed  to  itself,  the  MDSTC  of  the  requester 
sends  it  to  the  IS.  The  information  from  the  messages  Available  received  within  the 
timeout  period  will  be  used  by  the  IS  of  the  requester  to  select  a  number  of  sensors 
and  request  them  to  move.  The  selection  is  based  on  (i)  the  cost  per  unit  distance, 
(ii)  the  interval  of  time  necessary  to  move,  computed  using  the  speed  of  the  mobile 
sensor  and  its  distance,  and  (iii)  the  number  of  sensors  that  already  exist  in  the 
region  where  the  sensor  will  be  required  to  move,  which  we  call  it  obstruction. 

Let  C  be  the  region  of  interest  in  which  the  sensors  will  be  asked  to  move  by  a 
requester  node;  for  simplicity  we  assume  that  C  is  a  circle  of  radius  R.  Sensors  of 
the  desired  modality  will  be  placed  in  region  C,  which  is  included  in  the  requester 
communication  region,  in  order  for  the  requester  to  be  able  to  form  a  cluster.  In¬ 
stead  of  requiring  sensors  to  move  arbitrarily  to  any  location  in  C  we  propose  a 
uniform  placement  of  them.  To  this  end,  we  propose  two  methods  in  which  C  is 
partitioned  into  K  desired  regions  (see  Section  4.3).  For  each  of  the  available  sen¬ 
sors,  let  us  say  M  in  total,  we  compute  the  distance  between  the  node  and  each 
desired  region.  In  Section  4.4  we  present  a  Greedy  algorithm  that  selects  nodes  for 
each  of  the  K  regions,  using  the  M  x  K  such  distances. 

4.3.  Partitioning  a  Region  of  Interest  C 

We  propose  two  methods  of  partitioning  C  into  K  desired  regions: 

(i)  Concentric  Circles'.  Region  C  is  partitioned  into  K  equidistant  concentric  cir¬ 
cles  (see  Figure  3(a)). 

(ii)  Circular  Sectors'.  Region  C  is  divided  into  K  circular  sectors  (see  Figure  3(6)), 
of  angle  |"360/Ff]. 

We  compute  the  distance  a  node  (whose  modality  is  needed  by  the  requester 
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Figure  3:  Partitioning  the  communication  region  of  the  requester  into  K  desired  regions 

node  r)  has  to  move  in  order  to  reach  any  of  the  K  partitions  of  C.  For  any  node 
u,  let  Xu  be  the  location  of  node  u. 

4.3.1.  Concentric  Circles 

Region  C  is  partitioned  into  K  equidistant  concentric  circles  (see  Figure  3(a)). 
The  K  concentric  circles,  {Ci\i  =  1, . . . .  K),  create  K  —  1  rings  or  annulii  , 
{CSi \i  =  1, . . . ,  K  —  1},  with  the  cross-section  size  R/K\  the  radius  of  the  circle 
Ci  is  i  ■  ji.  By  abuse  of  notation,  let  CSq  be  the  circle  C\. 

We  will  compute  the  distance  a  node  whose  modality  is  needed  by  the  requester 
r  has  to  move  in  order  to  reach  any  of  the  K  regions.  Let  w  be  such  a  node  whose 
location  Xw  is  outside  C.  Let  d(Xw ;  CSi),  0  <  i  <  A'— 1,  be  the  distance  between 
Xw  and  the  region  CSi,  which  is  the  smallest  distance  the  node  w  needs  to  move 
in  order  to  touch  the  boundary  of  CSi.  Then 


d(Xw-CSi)  =  Xw-(i  +  l)R/K 


for  any  i,  0  <  i  <  K  —  1. 


An  annulus  is  the  region  between  two  concentric  circles  of  different  radii. 
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4.3.2.  Circular  Sectors 

We  divide  C  into  K  equal  circular  sectors  {Si,  i  =  1, . . . ,  K},  C  =  L)f=lSi 
(see  Figure  3(6)).  The  central  angle  of  each  sector  is  (3  =  For  simplicity,  we 
assume  that  360  is  divisible  by  K.  Each  Si  is  defined  by  the  center  Xr  and  the  two 
radii  R(i_\yp  and  Ri.p:  Si  =  (Xr;  R^-iyp',  R%.p)  3.  Note  that  two  consecutive 
circular  sectors  Sj  and  Sj+ 1  share  the  radius  Rj.p. 

Consider  some  node  w  whose  modality  is  desired  by  the  requester  r  and  whose 
location  Xw  is  outside  C.  Let  Dw  be  the  distance  from  Xw  and  Xr  and  aw  be  the 
angle  under  which  node  w  is  seen  by  node  r  (see  Figure  4). 


Figure  4:  Node  w  is  outside  C  and  needs  to  move  inside  C 

Let  d(Xw;Si),  1  <  i  <  K,  be  the  smallest  distance  between  Xw  and  any 
location  in  the  sector  Si,  which  represents  the  smallest  distance  node  w  needs  to 
move  in  order  to  touch  the  boundary  of  Si.  To  compute  d(Xw\  Si),  i ,  1  <  i  <  K, 
we  have  five  cases. 

Case  1)  aw  <  (i  —  1  )/3  and  (i  —  1  )/3  —  aw  <  |.  Let  7  =  (i  —  1  )/3  —  aw 
(Ligure  5 (a  —  6)).  We  have  two  subcases: 

Case  1.1)  Dw  COS7  >  R  (Ligure  5(a)).  The  closest  point  in  Si  to  Xw  is  the 
intersection  point  of  C  and  the  radius  R(i_i).p.  The  distance  node  w  needs  to  move 

3We  denote  by  Ra  the  ray  of  C  at  the  angle  a  from  the  x  axis. 
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in  order  to  reach  Si  is 


d{Xw\  Si)  =  \J (Dw  sin  7  -  R )2  +  (Dw  cosj)2. 

Proof.  The  distance  d(Xw;  Si)  is  the  length  a  of  the  edge  in  Figure  5(a),  and 
a  =  s/b 2  +  c2  where  b  =  Dw  sin  7  —  R  and  c  =  Dw  cos  7.  □ 

Case  1.2)  Dw  cos  7  <  R  (Figure  5(b)).  The  closest  point  in  St  to  Xw  is  on  the 
radius  R(i-\).p.  The  distance  node  w  needs  to  move  in  order  to  reach  Si  is 

d(Xw-Si )  =  Dw  sin 7. 

Case  2)  aw  <  (i  —  1  )f3  and  (i  —  l)/3  —  aw  >  |  (Figure  5(c)).  The  closest 
point  in  Si  to  Xw  is  Xr.  The  distance  node  w  needs  to  move  in  order  to  reach  St  is 

d(Xw ;  Si)  =  Dw  +  e 

where  e  >  0  is  a  value  close  to  0  and  is  needed  to  ensure  that  the  sensor  w  will  not 
be  placed  on  the  same  location  as  r. 

Case  3)  ( %  —  1  )@  <  aw  <i/3  (Figure  5(d)).  The  closest  point  in  Si  to  Xw  is  the 
intersection  point  of  C  and  the  radius  Raw .  The  distance  node  w  needs  to  move  in 
order  to  reach  S.t  is 

d(Xw ■  Si)  =  Dw  —  R. 

Case  4)  aw  >  i/3  and  aw  —  i(3  <  |.  This  case  is  similar  to  Case  1. 
d(Xw-Si)  =  sJ(Dw  sin 7  -  R)2  +  (Dw  COS7)2. 

Case  5)  aw  >  i/3  A  aw  —  i/3  >  |.  This  case  is  similar  to  Case  2. 

4.4.  Placement  Algorithm 

We  propose  Algorithm  Choose -1-out-of-M.  Executed  at  the  requester  node  r, 
it  assigns  to  each  circular  sector  an  available  sensor  with  the  lowest  cost.  Assign¬ 
ing  sensors  to  concentric  circles  can  be  done  in  similar  manner,  where  CSts  are 
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(a)  Case  1.1  (b)  Case  1.2 


(c)  Case  2  (d)  Case  3 


Figure  5:  Moving  node  w  into  the  sectors  of  the  desired  region  C 


considered  instead  of  ,5’(s.  The  proposed  algorithm  can  be  easily  extended  to  select 
I)  sensors  out  of  M  for  each  region,  or  to  assign  different  number  of  sensors  to 
different  regions. 

Algorithm  Choose- 1  -out-of-M  uses  the  following  variables.  For  each  sensor  i, 
1  <  %  <  M,  the  variable  e  {0, 1, . . . ,  K  )  indicates  either  the  circular  sector 
for  which  sensor  i  has  been  selected  {xi  >  1)  or  that  the  sensor  i  is  still  available 
(xi  =  0).  For  each  circular  sector  j,  1  <  j  <  K,  the  variable  yj  e  {0, 1, . . . ,  M} 
indicates  either  the  sensor  assigned  to  it  (yj  >  1)  or  that  the  circular  sector  is  still 
available  (yj  =  0). 

The  algorithm  has  at  most  K  +  1  steps.  Step  0  is  the  initialization.  The  x,s  and 
yj s  are  set  to  zero.  For  each  of  the  M  sensors  and  K  circular  sectors,  we  compute 
d(Xf,Sj),  the  minimum  distance  sensor  i  needs  to  move  in  order  to  reach  the 
boundary  of  the  desired  region  Sj,  following  the  method  described  in  Section  4.3. 
We  then  compute  cost(i ,  j),  the  cost  of  moving  sensor  i  to  region  Sj  as  the  product 
of  the  distance  d{Xi\  Sj)  and  the  cost  per  unit  distance  cpud(i)  of  sensor  i,  reported 
by  sensor  i  in  the  message  Available. 

At  each  step  k,  1  <  k  <  K,  the  algorithm  selects  an  available  sensor  that  has 
the  minimum  cost  for  some  region  that  has  not  been  occupied  and  invites  that  sen- 
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sor  to  move  in  that  region.  An  optimal  solution  for  Algorithm  Choose- 1-out-of-M 
seeks  to  select  moves  of  the  minimum  cost,  ignoring  the  moving  time.  Alterna¬ 
tively,  we  can  consider  moves  of  the  shortest  time  as  an  alternative  for  the  objective 
function;  in  that  case  we  will  compute  time(i,j)  to  be  the  time  it  takes  for  sensor 
i  to  move  to  region  ,Sj ,  which  is  simply  the  ratio  between  d(Xt:  Sj )  and  the  speed 
speed(i)  reported  by  sensor  i  in  the  message  Available. 

Theorem  2.  Algorithm  Choose  —  1  —  out  —  of  —  M  assigns  min{M ,  K}  sensors, 
one  per  circular  sector. 

Proof.  After  Step  0,  the  condition  xr  =  0  A  y:j  =  0  is  true  for  all  sensors  i, 
1  <  i  <  M,  and  all  circular  regions  j,  1  <j<K- 

In  Step  1,  the  instruction  LI  selects  the  minimum  cost  value  among  all  sensors 
and  all  circular  sectors.  The  sensor  zq  and  the  circular  sector  jo  with  that  minimum 
cost  will  be  assigned  to  each  other  by  the  instruction  L3.  Neither  the  sensor  zq  nor 
the  circular  sector  jo  will  be  considered  for  future  selection  by  the  instruction  L2 
executed  during  some  step  k,  1  <  k  >  K,  since  xU)  =  jo  and  jo  >  1,  and  yJ0  =  z0 
and  io  >  1. 

At  each  step  k,  1  <  k  <  K,  one  sensor  gets  assigned  to  a  single  circular  region, 
so  the  number  of  available  sensors  is  decremented  by  one.  If  M,  the  number  of 
sensors,  is  less  than  or  equal  to  K ,  then  the  instruction  LI  will  be  executed  exactly 
M  times  and  all  the  M  sensors  will  be  assigned  to  M  regions.  The  instruction  L2 
will  stop  the  algorithm  in  case  M  <  K.  If  M  >  K  then  the  instruction  LI  will 
be  executed  exactly  K  times  and  K  sensors  will  be  selected  and  assigned  to  K 
circular  sectors.  □ 

The  following  communication  occurs: 

-  On  receiving  one  or  more  messages  Available,  the  IS  decides  which  nodes  are  to 
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Algorithm  4.1  Algorithm  Choose  —  1  —  out  —  of  —  M 


Input: 

M 

total  number  of  sensors 

I< 

total  number  of  sectors 

Xr 

location  of  the  requester  node 

{Xi  — 

1  <  i  <  Mjlocation  of  sensor  i 

cpud{i ) 

,  1  <  i  <  M cost  per  unit  distance  of  sensor 

Step  0  ( Initialization ): 

for  i  =  1  to  M  do  set  Xi  =  0 
for  j  =  1  to  K  do  set  tjj  =  0 
for  i  =  1  to  M  do 
for  j  =  1  to  K  do 
compute  d(Xi]  Sj ) 

compute  cost(i,j)  =  d(Xf,  S  —  j)  x  cpud(i) 

Step  k,  1  <  k  <  K: 

LI  select  iq  and  jo  such  that 

cost(io,jo )  =  min{cost(i,  j)|(l  <  i  <  M)  A  Xi  =  0 
A(1  <j<K)  A  yj  =  0} 

L2  if  no  sensor  is  available  then  STOP 

L3  set  Xi0  =  jo;  set  y]Q  =  zq  //  assign  sensor  to  that  region 


be  invited  to  move  and  where  they  will  be  placed. 

-  The  MDSTC  of  a  node  that  receives  the  message  moveNode  not  addressed  to  it 
forwards  it  to  the  neighbor  that  has  forwarded  the  message  Available  to  it. 

-  On  receiving  the  message  moveNode,  the  MDSTC  of  the  desired  node  arranges 
for  the  node  to  move. 
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5.  Application  -  Tracking  a  Mobile  Target  in  an  Urban  Scenario 


In  an  urban  environment,  the  ambient  noise  for  certain  type  of  sensors  is  more 
prevalent  than  in  an  open  field  or  underwater  due  to  buildings,  people,  motor  ve¬ 
hicles,  and  so  on.  We  simulated  the  sensor  network  in  an  urban  environment  in 
NS-2,  using  sensory  data  collected  during  training  experiments  executed  in  our 
laboratory  for  locating  and  tracking  a  Segway  RMP  robot  (as  the  moving  object). 
Each  sensor  from  the  laboratory  has  been  simulated  in  NS-2  and  its  readings  were 
emulated  using  the  readings  of  that  sensor  in  the  lab  during  the  training  phase.  The 
spatial-temporal  information  of  the  Segway’s  movement  collected  by  individual 
sensors  are  fused  in  NS-2  by  clustering  the  sensors  along  the  estimated  path  of  the 
Segway.  The  cluster  heads  of  each  of  the  formed  clusters  then  estimate  the  Seg¬ 
way’s  position  and  velocity.  The  urban  scenario  in  NS-2  simulates  the  positioning 
of  the  sensors  in  the  lab  as  follows.  Three  types  of  sensors  were  deployed  on  roads 
arranged  as  a  Manhattan- like  grid  (see  Figure  6):  pressure  sensors,  video  sensors 
(cameras),  and  magnetic  sensors.  A  large  brown  block  represents  a  building  and  a 
yellow  dashed  line  is  a  road  marking  for  a  20-feet  wide  two-way  street. 
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Figure  6:  Urban  multi-modal  sensor  network 
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312  fixed  pressure  sensors  are  embedded  in  the  ground  of  two-way  streets. 
Each  pressure  sensor  generates  an  analog  voltage  due  to  the  pressure  applied  when 
an  object  goes  over  it.  This  voltage  is  fed  into  a  10-bit  A/D  channel  and  thereby 
the  output  ranges  from  0  to  1023.  Video  sensors,  mounted  on  the  buildings,  have 
adjustable  view  angles  so  that  the  sensing  coverage  can  be  easily  changed;  there  are 
six  of  them  on  each  building.  A  video  sensor  takes  snapshots  of  the  scene  within 
its  view  at  some  certain  frequency  and  these  images  are  used  to  construct  PFSA. 
72  magnetic  sensors  are  initially  positioned  close  to  the  street  intersections  and  are 
mounted  on  some  mobile  platforms  that  can  move  freely  along  the  streets;  they  can 
be  quickly  relocated  if  necessary. 

Nodes  communicate  wirelessly  using  the  two-ray  ground  reflection  radio  model. 
Each  sensor  node  uses  the  802.1 1  channel  access  scheme.  The  AODV  algorithm  is 
employed  as  the  routing  protocol  for  the  entire  network.  The  transmission  power 
of  a  single  node  is  0.3W  and  the  reception  power  is  0.2W.  We  recall  that  a  sensor 
broadcast  its  observed  subpattern  only  if  the  stimulus  created  by  the  target  is  above 
a  certain  threshold.  To  save  more  energy,  we  consider  a  sleep-and-awake  schedule 
for  the  sensor  nodes,  A-SAS,  proposed  by  [11].  For  a  1500  second-simulation, 
around  50%  percent  of  the  nodes  are  awake  at  any  moment.  Table  5  gives  the  en¬ 
ergy  consumption  of  the  nodes  when  A-SAS  is  used  or  not.  We  note  that  the  nodes 
that  have  joined  a  cluster  consume  more  energy  than  the  sensing  only  nodes.  We 
have  incorporated  and  run  the  rest  of  the  simulations  using  the  A-SAS  schedule. 


Sensing  only  nodes 

Nodes  once  in  a  cluster 

Without  A-SAS 

0.2194 

0.2905 

With  A-SAS 

0.1052 

0.2068 

Table  2:  Average  energy  consumption  per  sensor  node  during  a  1500  second-simulation  (in  Joules) 
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In  Section  3  we  showed  how  subpatterns  for  pressure,  magnetic,  and  video 
sensors  are  constructed  during  the  training  phase.  We  consider  composite  patterns 
that  are  3-tuple  of  subpatterns,  one  from  each  modality.  During  the  operational 
phase  we  locate  and  track  vehicles  carrying  large  amounts  of  metallic  material. 
Such  a  vehicle  will  trigger  subpattern  matches  at  the  pressure,  video,  and  magnetic 
sensors  that  are  geographically  closer  to  its  moving  location.  Thus  a  composite 
pattern  that  involves  PFSA  from  these  multi-modal  sensors  could  be  identified  and 
a  heterogeneous  cluster  that  contains  the  sensors  that  matched  the  subpattern  be 
formed.  Such  a  vehicle  is  deemed  by  our  sensor  network  as  suspicious  and  is 
marked  with  yellow  color  in  Figure  6.  Vehicles  that  do  not  carry  large  amounts  of 
metal  will  not  trigger  a  subpattern  match  at  any  magnetic  sensor,  thus  a  composite 
pattern  that  involves  PFSA  from  pressure,  video  and  magnetic  sensors  cannot  be 
identified.  Such  a  vehicle  is  considered  by  our  sensor  network  as  benign  and  is 
marked  with  green  in  Figure  6. 

Based  on  the  speed  of  the  robot  and  the  fact  that  a  cluster  needs  some  time 
to  form,  we  fix  the  maximum  lifetime  of  any  cluster  to  be  80  seconds  and  the 
minimum  lifetime  of  any  cluster  to  be  50  seconds. 

Target  location  estimation  is  done  at  the  cluster  head  after  the  cluster  is  formed. 
We  use  a  triangulation  scheme  based  on  the  location  of  the  cluster  members  to  esti¬ 
mate  the  target  location  which  works  as  follows.  When  a  cluster  has  been  formed, 
each  cluster  member  starts  estimating  its  distance  to  the  target  and  sends  that  dis¬ 
tance  to  the  cluster  head.  Then  the  cluster  head  collects  these  local  estimators  in¬ 
cluding  its  own  estimator  and  finalizes  a  localization  via  a  non-linear  least  squares 
triangulation  algorithm  [34].  Such  a  triangulation  algorithm  requires  at  least  three 
estimators  from  the  nodes  in  a  cluster.  In  general,  more  nodes  lead  to  better  local¬ 
ization.  If  the  size  of  a  cluster  is  less  than  three,  a  localization  cannot  be  properly 
done.  We  consider  only  that  the  magnetic  and  the  pressure  sensors  contribute  to 
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the  triangulation  for  heterogenous  clusters.  This  is  justified  on  the  basis  that  while 
a  video  sensor  may  be  more  important  for  long  distance  surveillance,  for  target  lo¬ 
calization  the  utility  of  magnetic  sensor  data  and  the  pressure  data  is  much  higher. 
Both  magnetic  and  pressure  sensors  are  assume  to  be  isotropic,  which  means  that 
the  sensor  measurement  depends  only  on  the  distance  to  the  target. 


Video  Sensor 
Pressure  Sensor 
Magnetic  Sensor 
Suspicious  Vehicle 
Benign  Vehicle 

Broadcast  Radius  of  the 
Cluster  Head 

Cluster  Member 

Cluster  Head 

Atomic  Pattern  Observed 


Figure  7:  Tracking  a  suspicious  vehicle  using  composite  patterns 

The  triangulation  scheme  used  by  the  cluster  head  has  a  smaller  error  than 
individual  estimations  sent  by  individual  sensor  nodes,  since,  without  clustering, 
the  localization  error  of  a  pressure  sensor  could  be  of  1  meter. 

Figure  7  shows  a  snapshot  of  the  NS -2  simulation  for  heterogeneous  cluster¬ 
ing.  Arrows  indicate  the  estimated  direction  of  the  target  (estimated  velocity)  and 
the  origin  of  the  arrow  indicates  the  estimated  position  of  the  target.  Flomogeneous 
clustering  using  the  same  sensor  network  is  shown  in  Figure  8;  three  homogeneous 
clusters,  corresponding  to  the  three  modality  sensors,  form  around  the  target.  How¬ 
ever,  only  the  cluster  of  pressure  sensors  generates  target  location  estimation  useful 
for  tracking,  because  the  other  two  clusters  do  not  have  good  location  estimation. 

Using  only  homogeneous  clusters,  it  is  impossible  to  do  a  proper  classification 
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Figure  8:  Snapshot  of  homogeneous  clustering 

when  two  targets  can  trigger  similar  subpatterns  for  some  modality  sensors  but 
not  for  all,  unless  we  add  another  layer  of  inter-cluster  communication.  Thus  the 
main  disadvantage  of  a  homogeneous  cluster  is  that  it  cannot  properly  distinguish 
benign  targets  and  malicious  targets,  except  for  the  cluster  of  magnetic  sensors, 
and  the  network  cannot  selectively  track  the  targets,  which  leads  to  unnecessary 
cluster  formation  and  therefore  wastes  the  network  resources.  In  Figure  7,  when 
two  vehicles  are  present,  one  labeled  as  ’’suspicious”  and  one  labeled  as  ’’benign”, 
the  benign  vehicle  carries  little  metal  and  will  not  trigger  subpattern  generation  at 
the  magnetic  sensors,  due  to  the  fact  that  its  magnetic  sensory  data  does  not  reach 
the  threshold  imposed  (see  Section  3). 

To  compare  the  localization  performance,  we  modify  MDSTC  to  form  either 
heterogeneous  or  homogeneous  clusters  in  two  scenarios:  (1)  homogeneous  clus¬ 
tering  ( homogJClus ),  (2)  heterogeneous  clustering  (heterJClus). 

For  both  scenarios,  only  one  target  is  included  since  an  homogeneous  cluster 
cannot  distinguish  between  the  benigna  and  malicious  target.  We  consider  different 
paths  for  the  target.  The  path  parameter  p  gives  the  ratio  of  the  distance  between 
the  target  and  the  right  building  and  the  width  of  the  road.  For  example,  when  p  is 
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0.5,  the  target  moves  exactly  in  the  middle  of  the  street.  We  range  p  from  0.4  to  0.6 
by  the  increment  of  0.05  in  the  simulation.  We  conduct  a  1500-second  simulation 
for  each  p  in  each  scenario. 

Figure  9  gives  the  histogram  of  the  number  of  clusters  formed  during  the  1 500- 
second  simulation  for  homogeneous  clusters  and  heterogeneous  clusters.  The  av¬ 
erage  number  of  clusters  is  around  ten  or  eleven,  while  in  the  case  of  homogeneous 
clustering  the  average  is  over  50. 


Number  of  Clusters  Number  of  Clusters 


(a)  Heterogeneous  clusters  (b)  Homogeneous  clusters 

Figure  9:  Histogram  of  the  number  of  clusters  formed 

Table  3  gives  the  average  error  of  position  estimation  for  various  path  param¬ 
eters  using  heterogeneous  and  homogeneous  clustering.  We  note  that  the  hetero¬ 
geneous  clustering  provides  a  better  localization  for  the  malicious  target  than  the 
homogeneous  one.  The  reason  is  that  in  the  heterogeneous  scenario,  the  number 
of  clusters  formed  is  smaller  than  the  homogeneous  scenario,  but  the  cluster  size 
is  larger.  A  larger  cluster  results  in  a  better  triangulation  estimation.  We  also  note 
that  for  some  path  parameters,  the  homogenous  clustering  does  not  give  any  es¬ 
timation.  This  is  due  to  the  fact  that  the  cluster  size  should  be  at  least  3  for  the 
triangulation  scheme  to  work  and  in  the  homogeneous  scenario,  either  the  mag¬ 
netic  sensors  are  geographically  far  away  from  each  other  or  the  pressure  sensors 
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have  a  relatively  short  sensing  range.  It  is  very  difficult  for  the  target  to  trigger 
three  magnetic  sensors  or  three  pressure  sensors  roughly  at  the  same  time. 


Scenarios  \  path  p 

0.4 

0.45 

0.5 

0.55 

0.6 

heterJJlus 

0.5917 

0.5644 

0.5443 

0.6050 

0.6822 

homogJJlus 

None 

0.7950 

0.6468 

None 

None 

Table  3:  Average  localization  estimation  error  in  meters  for  homogeneous  and  heterogeneous  sce¬ 
narios  and  various  path  parameter  values  (None  means  there  is  no  estimation  during  the  simulation) 

Table  4  gives  the  total  number  of  estimations  done  for  both  scenarios.  We  note 
that  the  heterogenous  clustering  does  many  more  estimations  than  the  homoge¬ 
neous  one  because  it  allows  different  types  of  sensors  to  be  included  in  one  cluster. 
Hence  our  heterogeneous  clustering  makes  the  tracking  more  robust. 


Scenarios  \  path  p 

0.4 

0.45 

0.5 

0.55 

0.6 

heterJJlus 

35 

28 

59 

11 

14 

homogJJlus 

0 

6 

39 

0 

0 

Table  4:  Number  of  estimations  done  in  both  scenarios  and  various  path  parameter  values 

All  three  types  of  sensors  must  be  present  in  the  vicinity  of  the  target  to  be 
able  to  form  a  heterogeneous  cluster.  If  the  sensor  field  is  not  dense  enough,  more 
likely  a  malicious  target  will  be  missed.  To  show  the  need  of  moving  sensors  and 
the  purpose  of  our  proposed  moving  algorithm,  we  purposely  put  some  magnetic 
sensors  to  sleep.  In  this  way  we  create  a  local  area  sparse  in  magnetic  sensors. 
As  shown  in  Figure  10,  sensors  number  5  and  34  (gray  dash  boxes)  are  sleeping 
and  no  magnetic  sensor  monitors  that  intersection.  Thus  the  track  of  the  malicious 
target  will  be  lost  when  it  reaches  that  intersection  since  no  cluster  will  be  formed 
without  at  least  one  magnetic  sensor.  We  have  implemented  the  first  method  of 
dividing  the  region  of  interest  using  concentric  circles  and  the  placement  algorithm. 
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That  will  trigger  the  magnetic  sensor  number  27  to  move  to  that  intersection,  as 
shown  in  Figure  1 1 .  For  our  simulations,  since  we  consider  that  every  composite 
pattern  needs  at  least  one  node  of  each  modality,  we  have  a  degenerated  case  of 
the  method  using  concentric  circles,  since  only  one  magnetic  sensor  is  requested  to 
move.  A  heterogeneous  cluster  is  formed  in  the  vicinity  of  the  intersection  around 
the  malicious  target  and  the  track  of  the  target  is  followed  further. 


Figure  10:  Sparse  sensing  area  at  some  intersection 
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Figure  1 1 :  A  magnetic  sensor  moves  to  a  sparse  sensing  area 


6.  Conclusion  and  Future  Work 

We  proposed  a  network  controller  that  is  responsible  for  clustering  multi-modal 
sensor  nodes  only  in  the  space-time  vicinity  of  a  moving  object,  thus  improving 
data  fusion  and  localizing  and  tracking  the  object  that  has  triggered  the  composite 
pattern.  We  showed  that  in  general  composite -base  estimation  is  more  accurate 
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than  individual  estimation,  i.e.  multi-modal  (heterogeneous)  clusters  offer  better 
estimation  than  homogeneous  clusters.  Thus  multi-modal  clusters  are  better  for 
estimation,  as  well  as  for  classification. 

Our  multi-modal  clustering  algorithm  MDSTC  considers  only  one-hop  clus¬ 
ters.  In  order  to  save  energy,  sensor  nodes  may  reduce  their  transmission  power 
so  they  may  need  to  form  multi-hop  clusters  in  order  to  group  enough  sensors  of 
various  modalities.  We  plan  to  use  the  work  of  [18,  19]  to  extend  the  results  of  an 
one-hop  cluster  to  a  multi-hop  cluster  where  the  transmission  schedule  of  the  nodes 
within  the  cluster  plays  a  much  more  important  factor  than  it  does  in  an  one-hop 
cluster.  We  plan  also  to  design  a  traffic  analyzer  that  observes  the  local  traffic  at  a 
node  in  order  to  predict  future  events  and  alter  the  transmission  schedule  based  on 
patterns  of  communication  detected  in  the  past  and  stored  locally  at  the  node. 
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